1. Field of the Invention
The present invention relates to a method and an apparatus for decoding the image signals encoded by using the orthogonal transform of the interframe differential signals.
2. Description of the Background Art
As a method of encoding image signals, there is a method which utilizes the orthogonal transform of the interframe differential signals of the image signals.
An example of a conventional image signal encoding apparatus using this method is shown in FIG. 1, where the encoding apparatus comprises: a subtractor 601 to which the input image signals are entered; an orthogonal transform device 602 connected with the subtractor 601; a quantizer 603 connected with the orthogonal transform device 602; an inverse quantizer 604 connected with the quantizer 603; an inverse orthogonal transform device 605 connected with the inverse quantizer 604; an adder 606 connected with the inverse orthogonal transform device 605; a frame memory 607 connected with the adder 606 through a switch 611; an encoder 608 connected with the quantizer 603, from which the encoded signals are outputted to a signal transmission line through a switch 610; a significance judgement device 609 connected with the subtractor 601, which controls the switches 610 and 611.
In this image signal encoding apparatus, the interframe differential signals are obtained by the subtractor 601 by subtracting the preceding frame image signals memorized in the frame memory 607 from the input image signals.
Then, the significance judgement device 609 determines each block within the frame as either one of a significant block or an insignificant block by evaluating the size of the interframe differential signals obtained by the subtractor 601 according to a prescribed evaluation algorithm. Each block is determined as the significant block when the size of the interframe differential signals is evaluated to be greater than a predetermined significant level according to the prescribed evaluation algorithm, whereas otherwise the block is determined as the insignificant block.
When a block is determined as the significant block, the significance judgement device 609 closes the switches 610 and 611, whereas when a block is determined as the insignificant block, the significance judgement device 609 opens the switches 610 and 611.
On the other hand, the orthogonal transform device 602 applies the orthogonal transform to the interframe differential signals obtained by the subtractor 601 to obtain the orthogonal transform coefficients.
Then, the quantizer 603 quantizes the orthogonal transform coefficients obtained by the orthogonal transform device 602, and the encoder 608 encodes the quantized orthogonal transform coefficients obtained by the quantizer 603 to obtain the encoded signals.
When the switch 610 is closed by the significance judgement device 609, the encoded signals obtained by the encoder 608 are outputted to a signal transmission line, whereas when the switch 610 is opened by the significance judgement device 609, the encoded signals obtained by the encoder 608 are discarded.
Meanwhile, the quantized orthogonal transform coefficients obtained by the quantizer 603 are entered into a decoding loop in which the inverse quantizer 604 inversely quantizes the quantized orthogonal transform coefficients so as to recover the orthogonal transform coefficients, and the inverse orthogonal transform device 605 applies the inverse orthogonal transform to the recovered orthogonal transform coefficients so as to recover the interframe differential signals.
Then, the adder 606 adds the recovered interframe differential signals and the preceding frame image signals memorized in the frame memory 607 so as to reproduce the current frame image signals. When the switch 611 is closed by the significance judgement device 609, the current frame image signals obtained by the adder 606 are entered into the frame memory 607 over the preceding frame image signals so as to be memorized in the frame memory 607 as the new preceding frame image signals to be utilized in the subsequent encoding operation of the input image signals for the next frame.
In this method of encoding image signals, a number of blocks to be decoded which are transmitted through the signal transmission line to an image signal decoding apparatus, i.e., a number of significant blocks in the encoding apparatus, varies in time according to the size of the dynamical changes appearing in the images to be encoded.
An example of a conventional image signal decoding apparatus for decoding the encoded signals obtained by the image signal encoding apparatus of FIG. 1 is shown in FIG. 2, where the decoding apparatus comprises: a decoder 701 to which the encoded signals are entered; an inverse quantizer 702 connected with the decoder 701; an inverse orthogonal transform device 703 connected with the inverse quantizer 702; an adder 704 connected with the inverse orthogonal transform device 703, from which the reproduced image signals are outputted; and a frame memory 705 connected with the adder 704.
In this image signal decoding apparatus, the decoder 701 decodes the encoded signals received from the image signal encoding apparatus through the signal transmission line so as to recover the quantized orthogonal transform coefficients, the inverse quantizer 702 inversely quantizes the quantized orthogonal transform coefficients so as to recover the orthogonal transform coefficients, and the inverse orthogonal transform device 703 applies the inverse orthogonal transform to the recovered orthogonal transform coefficients so as to recover the interframe differential signals.
Then, the adder 704 adds the recovered interframe differential signals and the preceding frame image signals memorized in the frame memory 705 so as to reproduce the current frame image signals. The current frame image signals obtained by the adder 704 are entered into the frame memory 705 over the preceding frame image signals so as to be memorized in the frame memory 705 as the new preceding frame image signals to be utilized in the subsequent decoding operation of the encoded signals for the next frame.
In this image signal decoding apparatus, the integration of the information on the interframe differences is carried out in the pel (picture element) domain, just as in the decoding loop in the image signal encoding apparatus of FIG. 1.
As a consequence, an error due to the difference between the mode of the integration of the information on the interframe differences in the decoding loop of the image signal encoding apparatus and in the image signal decoding apparatus can be avoided when the image signal decoding apparatus of FIG. 2 is used in decoding the encoded signals obtained by the image signal encoding apparatus of FIG. 1.
However, in this image signal decoding apparatus of FIG. 2, it is necessary to apply the inverse orthogonal transform to every encoded signal entered through the signal transmission line in order to achieve the integration of the information on the interframe differences. Yet, as already mentioned above, a number of blocks to be decoded which are transmitted through the signal transmission line to an image signal decoding apparatus varies in time according to the size of the dynamical changes appearing in the image contents encoded by the image signal encoding apparatus of FIG. 1, so that it is necessary in this image signal decoding apparatus of FIG. 2 to employ a high speed inverse orthogonal transform device capable of dealing with the peak value of the number of input blocks entering through the signal transmission line.
On the other hand, in a case the number of input blocks is significantly less than the peak value, it is quite unnecessary to fully utilize a high processing capability of such a high speed inverse orthogonal transform device, so that the processing operation in the image signal decoding apparatus with a high speed inverse orthogonal transform device is rather inefficient.
Another example of a conventional image signal decoding apparatus for decoding the encoded signals obtained by the image signal encoding apparatus of FIG. 1, which utilizes the image signal decoding method disclosed in Japanese Patent Application No. 2-46661, is shown in FIG. 3 where the decoding apparatus comprises: a decoder 801 to which the encoded signals are entered; an inverse quantizer 802 connected with the decoder 801; an adder 803 connected with the inverse quantizer 802; an inverse orthogonal transform device 804 connected with the adder 803, from which the reproduced image signals are outputted; an orthogonal transform coefficient memory 805 connected with the adder 803; and an inverse orthogonal transform device controller 806 to which the encoded signals are also entered, and which controls the inverse orthogonal transform device 804.
In this image signal decoding apparatus, the decoder 801 decodes the encoded signals received from the image signal encoding apparaus through the signal transmission line so as to recover the quantized orthogonal transform coefficients, and the inverse quantizer 802 inversely quantizes the quantized orthogonal transform coefficients so as to recover the orthogonal transform coefficients.
Then, the adder 803 adds the recovered orthogonal transform coefficients and the integrated orthogonal transform coefficients memorized in the orthogonal transform coefficient memory 805 in order to obtain the new integrated orthogonal transform coefficients which are subsequently entered into the orthogonal transform coefficient memory 805 over the previous integrated orthogonal transform coefficients so as to update the integrated orthogonal transform coefficients memorized in the orthogonal transform coefficient memory 805. The orthogonal transform coefficient memory 805 has a capacity to memorize the orthogonal transform coefficients corresponding to one frame of the image to be decoded.
On the other hand, the inverse orthogonal transform device controller 806 monitors a number of input blocks entering through the signal transmission line per unit time, and activates the inverse orthogonal transform device 804 when the monitored number of input blocks is less than a prescribed threshold value, such that the inverse orthogonal transform device 804 applies the inverse orthogonal transform to the new integrated orthogonal transform coefficients obtained by the adder 803 so as to reproduce the image signals, whereas when the monitored number of input blocks is greater than the prescribed threshold value, the inverse orthogonal transform device controller 806 stops the operation of the inverse orthogonal transform device 804.
Thus, in this image signal decoding apparatus, the information on the interframe differences is constantly integrated in the transform domain formed on the orthogonal transform coefficient memory 805, regardless of the number of input blocks entering through the signal transmission line.
In this image signal decoding apparatus, the image quality will be temporarily deteriorated while the operation of the inverse orthogonal transform device 804 is stopped, but the reproduction of the image signals can be resumed immediately when the operation of the inverse orthogonal transform device 804 is resumed.
Consequently, it is sufficient in this image signal decoding apparatus of FIG. 3 to employ a relatively low speed inverse orthogonal transform device because the operation of the inverse orthogonal transform device will be stopped whenever the number of input blocks per unit time is excessive.
However, an error due to the difference between the mode of the integration of the information on the interframe differences in the decoding loop of the image signal encoding apparatus and in the image signal decoding apparatus arises when the image signal decoding apparatus of FIG. 3 is used in decoding the encoded signals obtained by the image signal encoding apparatus of FIG. 1, and this error will be accumulated as the operation time of this image signal decoding apparatus progresses, so that the image quality obtainable by the image signals reproduced by this image signal decoding apparatus will steadily deteriorate as the operation time of this image signal decoding apparatus progresses.
Now, in an apparatus for receiving multiple channels of motion video signals such as a multi-point TV conference apparatus, it is necessary to decode the multiple channels of the motion video signals simultaneously. Here, if an exclusively devoted decoder is provided for each one of the multiple channels, the size of the decoding apparatus would have to be practically too large according to the present day VLSI technology. For this reason, it is preferable to share the decoder among a plurality of channels and utilize a concurrent decoding scheme for the decoding operation.
Conventionally, such an apparatus has a decoder simply shared among a plurality of channels, where the decoding operation is carried out for each channel by utilizing the same complete image signal decoding algorithm, so that the burden of the decoding operation for each decoder may have to be increased significantly.
As a consequence, it has conventionally been necessary to reduce the burden of the decoding operation at the decoding apparatus side by either thinning out the picture elements on the encoding apparatus side or selectively transmitting the encoded signals of the fraction of blocks within each frame.
However, in the case of the former, the blurring of the reproduced images due to the reduced number of picture elements occurs. On the other hand, in a case of the latter, the image quality of the reproduced images can be severely deteriorated by the buffer overflow occurring in a case of radical dynamical changes in the image contents such as scene changes in which case only a part of the image information can be transmitted.